A comparison of methods for dealing with troublesome examination questions.

This study compared two methods for identifying troublesome examination questions: a postexam student "challenge committee" process and a "question citation" procedure allowing students to identify troublesome questions during an exam. Challenged and cited questions on eight examinations given to second-year medical students were compared with respect to the percentage of correct responses (%CR) and the discrimination index (DI). Challenged questions had a significantly lower mean value for %CR (53.2 +/- 22.5, n = 74) than unchallenged questions (84.7 +/- 15.4, n = 428) but the two groups did not differ with respect to DI. For cited questions, the number of citations was correlated with %CR (r = -0.5480, p < 0.001) but not with DI. The group of questions that received three or more citations (n = 145) included 67.6% of all challenged questions and 78.3% of challenges that eventually led to a change in an answer key. A simple citation procedure completed during an exam could replace a complicated, post-exam challenge committee process and achieve the same goal of identifying troublesome questions.